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Abstract 

We consider the problem of estimating a conditional covariance matrix in an in- 
verse regression setting. We show that this es timation can be achieved by estimating 
a quadratic functional extending the results of Da Veiga & Gamboal (120081) . We prove 
that this method provides a new efficient estimator whose asymptotic properties are 
studied. 



1 Introduction 

Consider the nonparametric regression 

Y = <p(X) + e, 

where X e K p , FGl and E [e] =0. The main difficulty with any regression method is that, 
as the dimension of X becomes larger, the number of observations needed for a good esti- 
mator increases exponentially. This phenomena is usually called the curse of dimensionality. 
All the "classical" methods could break down, as the dimension p increases, unless we have 
at hand a very huge sample. 

For this reason, there have been along the past decades a very large number of methods 
to cope with this issue. Their aim is to reduce th e dimensionality o f the problem, using just 
to name a few, the genera lized linear model in iBrillingerl (11983b . the additive models in 
Hastie & Tibshirani (1 1990 ). sparsitv constraint models as Q ( 20071) and references therein. 

Alternatively. IlJ ( 1991a ) proposed the procedure of Sliced Inverse Regression (SIR) 
considering the following semiparametric model, 

Y = <t>{vJX,...,v T K X,e) 

where the v's are unknown vectors in W, e is independent of X and 4> 1S an arbitrary 
function in M. K+1 . This model can gather all the relevant information about the variable 
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Y, with only the projection of X onto the K <C p dimensional subspace (vJX, . . . , v^X). 
In the case when K is small, it is possible to reduce the dimension by estimating the i/s 
efficiently. This method is also used to search nonlinear structures in data and to estimate 
the projection directions v's. For a review on SIR methods, we refer to ID fll991ailbh: Duan 



& Li (119911) : lHardle & Tsybakovl (119911) and references therein. The v's define the effective 
dimension reduction (e.d.r) direction and the eigenvectors of E[Cov(X|Y)l are the e.d.r. 
directions. Many estimators have been proposed in or der t o study the e .d.r d i rectio ns in 
many different ca ses. Fo r exam ple, Izhu & Fangl (1 19961) and lFerre & Yaol (120051. 2003) use 
kernel estimators, iHsingi (1 19991) combines neare st neighbor and S IR, Bura & Cook! (bOQlT) 
assume that EjXj Y] has some parametric form, ISetodji & Cook! (12004) use k-means and 
Cook & Nil (120051) transform SIR to least square form. 

In this paper, we propose an alternate estimation of the matrix 

Cov(E[X|Y]) = E[E[X|r]E[X|r] T ] -E[X]E[X] T , 



using ideas developed by lDa Veiga & Gamboal (120081) . inspired by the prior work of lLaurent 
(Il996l) . More precisely since E[X]E[X] T can be easily estimated with many usual meth- 
ods, we will focus on finding an estimator of E[E[X|F]E[X|F] T ] . For this we will show 
that this estimation implies an estimation of a quadratic functional rather than plugging 
non parametric estimate into this form as commonly used. This method has the advantage 
of getting an efficient estimator in a semi-parametric framework. 

This paper is organized as follows. Section[2]is intended to motivate our investigation of 
Cov(E[X|Y] ) using a Taylor approximation. In Section [3TT1 we set up notation and hypoth- 
esis. Section [3721 is devoted to demonstrate that each coordinate of Cov(E[X|F]) converge 
efficiently. Also we find the normality asymptotic for the whole matrix. An asymptotic 
bound of the variance for the quadratic part for the Taylor's expansion of Cov(E[X|Y]) is 
found in Section 0} All technical Lemmas and their proofs are postponed to Sections [6] and 
[5] respectively. 



2 Methodology 

Our aim is to estimate Cov(E[X|Y]) efficiently when observing X e IR P , for p>l, and 
Y e R. For this , write the matrix 

Cov(E[X|Y]) =E[E[X|F]E[X|r] T ] -E[X]E[X] T , 

where A T means the transpose of A. If E[X] can be easily estimated by classical methods, 
the remainder term 

E[E[X|F]E[X|F] T ] = (7~).. /../ 1 ,k 

is a non linear term whose estimation is the main topic of this paper. Each term of this 
matrix can be written as 



f f(x i ,x j ,y)dx i dx j J \ J f(x i ,x j ,y)dx i dx j 



f(x i ,x j ,y)dx i dx j dy, (1) 
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where f{xi, Xj, y) for i and j fixed, is the joint density of (X i: Xj,Y) i,j = 1, . . . , p. 

Hence, we focus on the efficient estimation of the corresponding non linear functional 

for / G h(dxi, dxj, dy) 



oc 2 j~ ^ oc 2 2 oc j • f y ^ doc 2 doc j ^\ ^* 3> ^ ^* ^ 2 ^ «3j ^ - ^ doc 2 doc j 



f H- (/) = / ( ' ' 1 ( ff(g , ;. ; )da ,, d3 ,. ) ffax^dxidxjdy. 



(?) 



In the case i = j, this estimation has been considered in |Pa Veiga & Gamboal (|2008) : 
Laurent ( 19961) . Here we extend their methodology to this case. Assume we have at hand 
an i.i.d sample (X- k \ Xj\ Y^), k — 1, . . . , n such that it is possible to build a preliminary 

estimator / of / with a subsample of size n\ < n. Now, the main idea is to make a 
Taylor's expansion of Ty(/) in a neighborhood of / which will play the role of a suitable 
approximation of /. More precisely, define an auxiliar function F : [0, 1] — > W; 

F(u)=T ij (uf + (l-u)f) 

with u G [0,1]. The Taylor's expansion of F between and 1 up to the third order is 

F(l) = F(0) + F'(0) + V(0) + \f>"{£){1 - 3 (3) 

for some £ G [0, 1]. Moreover, we have 

F{l)=T lj {}) 

f(o) = Tlj (f) = f ( f*ifci>*i>y)<toi*°i\ ( fxifcux^dxidxA f{x ^ y)dXidx . dy _ 



f f(x i ,x j ,y)dx i dx j J \ f f(x i ,x j ,y)dx i dx J 
To simplify the notations, let 



mi(fu,y) 
mi(fo,y) = rrii(f,y) 



J Xif u (Xi : Xj : y}dX{dx 



J 



J fu {xi > Xj , y)dx{dxj 
J Xj / f{xj i ^Xj^y^dxj i dxj 



J f(.Xi-, y^dx^dxj 
where f u — uf + (1 — u)f, Vu G [0, 1]. Then, we can rewrite F(u) as 

F(u) = / m i (f u ,y)mj(f Uj y)f u (xi,Xj J y)dx i dx j dy. 



The Taylor's expansion of Ty(/) is given in the next Proposition. 

Proposition 1 (Linearization of the operator T). For the functional Tij(f) defined in §2$, 
the following decomposition holds 

T ij(f) = J H 1 (f,x i ,Xj,y)f(x i ,Xj,y)dx i dx j dy 

+ J H2(f,Xi 1 ,Xj2,y)f(xii,x j x,y)f(xi2,x j 2,y)dx il dx j xdxi2dxj2dy + r n (4) 
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where 



Hi(f,Xi,Xj,y) = x i m j (f,y) + x j m i (f,y) - m % (f ,y)rrij(f ,y) (5) 
H 2 (f,x a ,x j2 ,y) = —j (xn - rriiif, y)) (x j2 - mj(f, y)) (6) 

T n = V(0(l-0 3 , (7) 
o 

for some £ e]0, 1[. 

This decomposition has the main advantage of separating the terms to be estimated 
into a linear functional of /, which can be easily estimated and a second part which is a 
quadratic functional of /. In this case, Section [4] will be dedicated to estimate this kind of 
functionals and specifically to control its variance. This will enable to provide an efficient 
estimator of T^-(/) using the decomposition of Proposition [TJ 



3 Main Results 

In this section we build a procedure to estimate 7y (f) efficiently. Since we used n\ < n to 
build a preliminary approximation /, we will use a sample of size n 2 = n — rti to estimate 
© and ©. Since ([5]) is a linear functional of the density /, it can be estimated by its 
empirical counterpart 

^-Y,Mf^ ( t\xf\Y^). (8) 

k=l 

Since © is a nonlinear functional of /, the estimation is harder. Its estimation will be 
a direct consequence of the technical results presented in Section [4] where we build an 
estimator for the general functional 



®{f) = J V( x a, x j2,y)f(xii,x j x,y)f(xi2,xj2,y)dxi 1 dx j xdxi2dxj 2 dy 
where 77 : R 3 — > R is a bounded function. The e stimator 9 n of 6(f) is an extension of the 



method developed in lDa Veiga & Gamboal (12008) 



3.1 Hypothesis and Assumptions 

The following notations will be used throughout the paper. Let d s and b s for s = 1,2,3 
be real numbers where d s < b s . Let, for i and j fixed, ~h 2 (dxidxjdy) be the squared inte- 
grable functions in the cube [di,&i] x [d 2l b 2 ] x [c/ 3 ,6 3 ]. Moreover, let xj, y))i &D be an 

orthonormal basis of \?{dxidxjdy), where D is a countable set. Let a t = f p t f denote the 
scalar product of / with p t . 

Furthermore, denote by iJ(dxidxj) (resp. ~h 2 (dy)) the set of squared integrable func- 
tions in [di,6i] x [d 2 ,b 2 ] (resp. [d 3 ,6 3 ]). If (a la (x l ,x j )i aeDi )(resp. (Pi p (y)i peDa )') is an or- 
thonormal basis of h 2 (dxidxj) (resp. h 2 (dy)) then pi(x h Xj,y) = 

®la \ X ij X 

j)P h (y) with 

I = (LJ/s) GAx d 2 . 
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We also use the following subset oiL, 2 (dxidxjdy) 

£ = I em '■ ( e ')'e^ is such that 

UeD leD 

where (ci)i eD is a given fixed sequence. 

Moreover assume that (X i} X,-, Y) have a bounded joint density / on [di, bi] x [d 2 , b 2 ] x 
[d 3 , b 3 ] which lies in the ellipsoid S. 

In what follows, X n — > X (resp. X n — > X) denotes the convergence in distribution or 
weak convergence (resp. convergence in probability) of X n to X. Additionally, the support 
of / will be denoted by supp /. 

Let (M n ) n>1 denote a sequence of subsets D. For each n there exists M n such that 
M n c D. Let us denote by \M n \ the cardinal of M n . 

We shall make three main assumptions: 

Assumption 1. For alln > 1 there is a subset M n c D such that (sup^ Mn |q| 2 ) 2 ps |MJ /n 2 
64 n ps -B means Ai < A n /B < \ 2 for some positives constants Ai and A 2 J. Moreover, V/ G 

h 2 (dxdydz), J (S M J - ff dxdydz -> when n ->■ 0, where S Mn / = £j eA / n 

Assumption 2. supp/ c KA] x [^2,^2] x [^3,^3] V(x,y,z) G supp/, < a < 
f{x,y,z) < /3 with a, (3 G R. 

Assumption 3. Jt is possible to find an estimator f of f built with n-y ps n/log(n) observa- 
tions, such that for e > 0, 

V(x, y, z) G supp /, < a - e < /(x, y, z) < (3 + e 

and, 

V2 < q < +00, V/ G N*, E / ||/-/|| i < C(q,l)n^ a 
for some A > 1/6 and some constant C(q, I) not depending on f belonging to the ellipsoid S. 

Assumption [1] is necessary to bound the bias and variance of 6 n . Assumption [2] and [3] 
allow to establish that the remainder term in the Taylor expansion is negligible, i.e T n = 
0{l/n) . Assumption [3] depends on the regularity of the density function. For instance for 
x G W, s > and L > 0, consider the class H q (s,L) of Nikol'skii of functions / G U{dx) 
with partials derivatives up to order r = inclusive, and for each of these derivatives 

Then, Assumption [3] is satisfied for / G 1-L q {s, L) with s > |. 
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3 . 2 Efficient Estimation of Ty ( / ) 

As seen in Section |2j can be decomposed as ©. Hence, using d8]) and ([141) we 

consider the following estimate 



J n 2 L — ' 

fc=i 

n( ^E E wC^.xf.yW) f B ( I , I ^ ) yW)^(/ I ^ lI ,^ ) xf ) yW) iB4(itj 

2^2 J JeMfe^fe'=l ^ 



zy z ' l,l'eMk^k'=l 

Pi(xa, xji, y)pi'(x i2 , x j2 , y)H 2 (f, xa,x j2 , y)dx il dx jl dxi2dx j2 dy. 

where H 3 (f,x il ,x jl ,x i 2,x j 2,y) = H 2 (f,xa,Xj 2 ,y) + H 2 (f,x i2 ,x jl ,y) and n 2 = n - n x . The 
remainder r n does not appear because we will prove that it is negligible when compared 
to the other error terms. 

The asymptotic behavior of for i and j fixed is given in the next Theorem. 

Theorem 1. Let Assumptions \Jj3\hold and \M n \ jn — > when n — > oo. Then: 

^(f^-T^f)) A ^(0,C y (/)), (9) 

and 

lim nEpf™ - T 4J (/)] 2 = Cy(/), (10) 

where 

C lJ (f)=V a i(H 1 (f,X i ,X v Y)) 

Note that, in Theorem (TJ it appears that the asymptotic variance of Tij(f) depends 
only on H 1 (f,X i ,Xj,Y). Hence the asymptotic variance of T>^ is explained only by the 
linear part of (|4]) . This will entail that the estimator is naturally efficient as proved in the 
following. 

Indeed, the semi-parametric Cramer-Rao bound is given in the next theorem. 
Theorem 2 (Semi-parametric Cramer- Rao bound.). Consider the estimation of 

J Xif(xijXj,y^dxidxj\ f J Xjf{xi^Xj^y^dx^dXj 



Ti-(f) = / — _ _ _ a; j j j j v ~„ f( XhX -,y)dXidxdy 
13 J V / f(x i ,x j ,y)dx i dx j J \ J f(x i ,x j ,y)dx i dx j J v 13 

for a random vector (X i; Xj, Y) with joint density f e £. Let f e£bea density verifying the 



assumptions of Theorem^ Then, for all estimator T^' ofTij{f) and every family {V r (/ )} 
of neighborhoods of f we have 



r>0 



inf liminf sup nE[l$N - T v {f )] A > C v {f ) 

{Vr.(/ )}r>0 ^OO / G V r (/ ) 



where V r (f ) = {/ : ||/- /o|| 2 < r} for r > 0. 
Consequently, the estimator T^y is efficient. 



In the case of our estimate, its variance is Cy(/), which proves its asymptotically effi- 
ciency. 

Remark that Theorem [1] proves asymptotic normality entry by entry of the matrix 
T(f) = (Tij(f))p Xp . To extend the result for the whole matrix it is necessary to intro- 
duce the half-vectorization operator vech. This operator, stacks only the columns from the 
principal diagonal of a square matrix downwards in a column vector, that is, for an p x p 
matrix A = fa,- 



vech(A) = [an, • • ■ , a pl , a 22 , - • • , a p2 , • - • , a 33 , ■ • • , a pp 



Let define the estimator matrix T™ = (T^ n) ) and denote the matrix with entries 

(Hi(f, Xi, Xj,y)) i -. Now we are able to state the following 

et, 

following properties 



-~(n) 

Corollary 1. Let Assumptions [T||3] hold and \M n \ /n — > when n — > oo. Then T has the 



v^vech(f (n) -T(/)) A^(0,C(/)), (11) 
lim nE[vech(f W - T(f)) vechff (n) - T(f)) T ] = C(f) (12) 

moo L v ' v ' J 

where 

C(f) = Cov(vech(Hi(/))) 

Previous results depend on the accurate estimation of the quadratic part of the estimator 
of which is the issue of the following section. 

4 Estimation of quadratic functionals 

As pointed out in Section |2] the decomposition © has a quadratic part © that we want to 
estimate. To achieve this we will construct a general estimator of the form: 

e = / ,(*,,«„, »)/(*,, «,., »)/(*■, y)^,^ i2 dy, 

for / e £ and 77 : M 3 — > K a bounded function. 
Given M n a subset of D, consider the estimator 

n 

^ = - r J - rf E E M^fUf,^) 

nm — 1 / — ' ^— ' ^ 

Pl ( Xi , Xj , xf '\ + 7?(xf } , Xj , y^)) d Xi d Xj 



^EE p l {X?\xf\YW) Pv {x\ k \xf'\Y^) 
' v eM fe^fe'=i 

Xji, y)pi'(x i2 , x j2 , y)r)(xa, x j2 , y)dx il dx j idx i2 dx j2 dy. (13) 



n\n 
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In order to simplify the presentation of the main Theorem, let ip(x il ,Xji,x i i,Xj2,y) 
r](xa, Xj 2 , y) + 7](x i2 , x jh y) verifying 



ijj(xi 1 ,x jl ,Xi2,x j 2,y)dx il dx jl dxi2dx j2 dy = j ip(x i2 , x j2 , x a , x jX , y)dx il dx jl dxi2dx j2 dy. 
With this notation we can simplify (fT3l) in 



On 



n\n 



^titE E vi{x<t\xf\Y^) / , w (x i) x iJ y^(x i) x i) x^ J x^ ) yW)dx i dx j 

' l£Mk^k'=l J 
1 n 

7 7T E E P l {x\ h \xf\Y^) Pv {xf\xf'\Y^) 

v y !,I'£M fc^fc'=l 

»(*a, ri^d^^. (14) 

Using simple algebra, it is possible to prove that this estimator has bias equal to 



- J {S M f{x il ,x j x,y) - f(xii,Xji,y))(S M f(xi2,Xj2,y) - f(x i2 ,x j2 ,y)) 

r}(xn, Xj2, y)dxiidx jl dxi2dx j 2dy (15) 

The following Theorem gives an explicit bound for the variance of n . 

Theorem 3. Let Assumption \T\ hold. Then if \M n \/n — > when n — > 0, then 6 n has the 
following property 

nE[(6 n -e) 2 ] -A(f,rj) 



<7 



\M„ 



n 



\SM n f - /|L + \\S Mn 9 - g\ 



(16) 



where g(x h Xj, y) = f f(x i2 , x j2 , y)i/){xi, Xj, x i2 , x j2 , y)dxi2dx j2 and 



Mf,v)= / g(xi,Xj,y) f(x i ,x j ,y)dx i dx j dy - / g(x i ,x j ,y)f(x i ,x j ,y)dx i dx j dy 



where 7 is constant depending only on WfW^, \\' n \\ 00 > an d ^ Xi xj = (&i ~~ a i) x (P2 — ^2)- More- 
over, this constant is an increasing function of these quantities. 

Note that equation (fT6l) implies that 

lim nE[9 n -6] 2 = A(f,rj). 
These results will be stated in order to control the term 

Q = H2{f,x il ,x j 2,y)f{x il ,x jl ,y)f{x i 2,x j 2,y)dx il dx j idx i 2dx j 2dy 



which has the form of the quadratic functional with the particular choice r}(xn,Xj2 : y) = 
H2(f,Xii,Xj 2 ,y). We point out that we also show that in this particular frame, we get 



A(/, 77) = 0. This the reason why the asymptotic variance of the estimate 7 1 / built in the 



previous section, is only governed by its linear part, yielding asymptotic efficiency. 
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5 Proofs 



Proof of Proposition \T\ 

We need to calculate the three first derivatives of F(u). In order to facilitate the calcu- 
lation, we are going to differentiate rrii(f u , y): 

d , . d f I Xif u yXi } Xj } y)dx{dxj 

-T- (mifu, y)) = T[ —rr~( \a a 

du du \ J j u {Xi,Xj,y)dXidXj 

_ f Xj(f(xj, Xj, y) - fjxj, xj, y))dxidxj 

J ^ifufaii y^)dxidxj J f[xi, Xjj ?/) /(x^, Xj, y^dxidxj 
(J fu(x i ,x jj y)dx i dx j ) 2 

J •£«(,/' (*^*' v) f j i yY)dxidx j 

J fu^it ■Eji y)dxidxj 
miifu, y) f f(x u Xj, y) - f(x h Xj, y)dxjdxj 
J fu 5 Xj , y^dxidx j 
J(xi- mi(f u ,y))(f(xi,Xj,y) - f{x h Xj,y))dXidxj 



J fu(.Xi } Xj, y)dxidx j 
Now, using (fTTl) we first compute F'(u), 

^ (rrii(f u , y)) mj(f Uj ,y)f u (xi, Xj, y) + mi(f u , y)-^- (rrij(f u , y)) f u (x h Xj, y) 
+rrii(fu, y)rrij(f u , y)-^ (f u (xi, Xj, y)) dxidxjdy, 



(17) 



x 



™j{fu, y) + Xjiriiifu, y) - mi(f w y)rrij(f u , y)\ (f(x h Xj, y) - f(x h x j: y))dxidxjdy. 



Taking u = we have 

^'(0) = J x i m j (f,y) + x j m i (f,y)-m i (f,y)m j (f,y) (f(x h Xj,y) - f(x u Xj,y))dxidxjdy. 

(18) 

We derive now m;(/ u , y)m,j(f u , y) to obtain 
^ (mi(fu,y)mj(f u ,y)) = ^ (mi(f u ,y))mj(f u ,y) + m i(f^y)-^ ( m j(fu,y)) 



mj{fu,y) 



f(xi - mi(f u ,y))(f(xi,Xj,y) - f(x h Xj,y))dxidxj 



+ mi{fu,y) 



J fu(xijXjjy)dxidxj 
f(xj -mj(f u ,y))(f(xi,Xj,y) - f(x l ,x j ,y))dx i dx j 



J fu(xi , Xj , y^dxidx j 



(19) 
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Following with F"(u) and using ( fTTD and ( fT9l ) we get, 

/( gj 2 - mj(f u ,y))(f{xi 2 ,Xj2,y) - f(xi2,x j2 ,y))dx i2 dx j2 
J fu^ii-^jiV^dxidxj 

J (x i2 - rrii(f u , y)) (f(x i2 , x j2 , y) - f(x i2 , x j2 , y))dx i2 dx j2 



F"(u) 



Xil 



+ Xji 



-mj(f u ,y) 



™>i{fu,y) 



J fui-^ii Xj^y^dxidxj 
J(x i2 -mi(f u ,y))(f(xi2,Xj2,y) - f{x i2 ,x j2 ,y))dxidxj 

J fu {x% i Xj , y^dx^dxj 
f(x j2 - m j (f u ,y))(f(x i2 ,x j 2,y) - f(x i2 ,x j2 ,y))dx i2 dx j2 



J fu(xi.,Xj,y^dxidxj 
(f{x a , x n , y) - fixi^Xj^y^dxadxjxdy. 

Simplifying the last expression we obtain 

F"{u) = 



{(x a -rrii(f u ,y)) (x j2 - mj(f u ,y)) + (x i2 - m i (f u ,y))(x j i - mj(f u ,y))} 



J fui^XtiXjjy^dxidxj 

(f{x a , x n , y) - f(x a , Xj!, y)) (f(x i2 , x j2 , y) - f(x i2 , x j2 , y))dxiidx jX dx i2 dx j2 dy. 

Besides, when u — 

F"(0) = (20) 
—j 1 { (x a - im(f, y)) (x j2 - rrijif, y)) + (x i2 - m;(/ , y)) (x n - m^f, yj) } 

(f(x a , Xji,y) - f(x a , x jh y)) (f(x i2 , x j2 , y) - f(x i2 , x j2 , y^dxudx^dx^dx^dy 

= j yj, — 2 w i x a - m i(f> v)) { x j2 - m j(f, y)) 

J J \Xi) x j i y jttXidx j 

(f(xa,x jh y) - f(xa,Xji,y)) (f(x i2 ,x j2 ,y) - f(xi 2 ,x j2 ,y))dxiidx jl dx i2 dx : j 2 dy. (21) 
Using the previous arguments we can finally find F"'{u): 

F"'(u) = 

I J f^(x i ,x^y)dx i dx j ^" " "*<'•■ »» ~ ^» ^ 
(f(xa, x jU y) - f(xa, x jU y)) (f(x i2 , x j2 , y) - f(x i2 , x j2 , y)) 

(f(x i3 , x j3 , y) - f(x i3 , x j3 , y^dxadxjxdxadxpdXiadXjzdy (22) 

Replacing (fT8l) . (12TT) and (1221) into d3]) we get the desired decomposition. □ 
Proof of Theorem [T| 
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We will first control the remaining term (O 

1 



r n = -F'"(0(i-0 3 - 

o 



Remember that 



. (x a - rrii{fz, y)) (x j2 - m^fe, y)) . 
F (0 = -6 / — — 7^T \j{.Xi\,Xji,y) ~ j{xn,Xji,y) 



(f f(,(x i ,x j ,y)dx i dx j )' 
f(x i2 , x j2 , y) - f{x i2 , x j2 , y)) (f{x i3 , x j3 , y) - f(x i3 , x j3 , y) ) dx il dxj 1 dxi 2 dx j2 dx i3 dx :j3 dy, 



Assumptions [T]and [2] ensure that the first part of the integrand is bounded by a constant /i. 
Furthermore, 



|r n | <n 



f(x a , x jU y) - f(x a , Xjt, y) f(x i2 , x j2 , y) - f(x i2 , x j2 , y) 



f(x i3 , x j3 , y) - f(x i3 , x j3 , y) dx il dxj X dx i2 dxj 2 dx i3 dxj 3 dy 



f(xi,Xj,y) - f(xi,Xj,y) 



dxidxj J dy 



f(xa,Xji,y) - f{x iU x jU y) 



dxidxjdy 



by the Holder inequality. ThenE[rg = 0(E(J \f - f\ 3 ) 2 ) = 0(E\\f - /|||). Since / verifies 
Assumption [3j this quantity is of order 0(n^ 6X ). Since we also assume n x « nj log(n) and 
A > 1/6, then n^ 6A = o (i) .Therefore, we get E[r^] = o(l/n) which implies that the 
remaining term T n is negligible. 

To prove the asymptotic normality of T-?\ we shall show that y/n (t^ — T^(/)J and 
define 



i " 2 r 

4 = -J2Mf^ X t\^ k) ,Y^) - / H^f^x^fixi^yVdXidXjdy (23) 
U2 k=i J 

have the same asymptotic behavior. We can get for z\y a classic central limit theorem with 
variance 

C ij(f) = Var(ifi(/,x i ,x j ,?/)) 



Hi(f, Xi, xj, y) 2 f(xi, xj, y))dxidxjdy 



Hi(f, Xi, xj, y)f(xi, xj, y))dxidxjdy 



which implies ([9]) and (fTOl) . In order to establish our claim, we will show that 
has second-order moment converging to 0. 



(24) 
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Define z\f as z\f with / replaced by /. Let us note that B$ = R x + R 2 where 



Ri 



Ri 



T, 



00 



i.) 



?(n) 



J i3 



It only remains to state that E[i2f] and E[i2|] converges to 0. We can rewrite R± as 



Ri 



Q-Q + T r 



where we note that 



Q = J H 2 {f, x a , x j2 , y)f{xa, x jU y)f{x i2 , x j2 , y)dx il dx jl dx i2 dx j2 dy 



H 2 (f,x a ,x j2 ,y) 



Xii-mi(f,y)) [x j2 -mj(f,y) 



has the form of a quadratic functional studied in Section@]with rj(xa,Xj 2 , y) = H 2 (f, Xn,Xj 2 , y). 
Hence such functional can be estimated as done in Section [4] and let Q be its corresponding 
estimator. Since E[r^] = o(l/n), we only have to control the term y/n(Q — Q) which is 

such that Hindoo nE [Q — Q] 2 = by Lemma This Lemma implies that E \Rf\ — > as 
n — > oo. For R 2 we have 



E[R 2 2 ] 



n 



n 2 



n 
n 2 



Hi(f, x h Xj, y) - Hi(f, x h Xj, y)J f(x h Xj, y))dxidxjdy 
H^f, Xi, Xj,y)f{xi, Xj, y))dxidxjdy - / H x (f, x h Xj, y) 2 f{x h Xj,y))dxidxjdy 



The same arguments as the ones of Lemma [7] (mean value and Assumptions [2] and [3]) show 
thatE[i?!] ^o. □ 

P roof of Theorem\2\. To prove the inequality we will use the usual framework described 
in libra gimov & Khas'minskiil (Il99ll) . The first step is to calculate the Frechet derivative of 



Tij(f) at some point f e S. Assumptions [2] and [3] and equation ((4]), imply that 



f(xi, xj, y) - f (Xi, Xj, yyjdXidXjdy + O ( / (/ - /„)' 



where rrii(f ,y) = J Xifo(xi,Xj,y)dxidxjdy / J fo(xi,Xj,y)dxidxjdy. Therefore, the Frechet 
derivative of T^f) at f is 7?.(/ ) ■ h = (H^fo, ■), h) with 

Hi(fo,Xi,Xj,y) = x i m j (f Q ,y) + x j m i (/ ,y) - mi(f ,y)mj(f ,y). 
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Using the results of llbragimov & Khas'minskiil (Il99ll) . denote 

H(fo) = {u £ L?(dxidxjdy), f u(xi, Xj, y)\J fo{xi, Xj, y)dxidxjdy = 0} the set of functions 
in ~L 2 {dxidxjdy) orthogonal to v7o, P r -ff(/o) tne projection onto H(f ), A n (t) = (y/Jo)t/y/n 
and Pj n) the joint distribution of (xf \ xf ] ) k = 1, . . . , n under /„. Since (x\ k \ xf ] ) k = 

1, . . . ,n are i.i.d., the family {P^\ f G £} is differentiable in quadratic mean at / and 
therefore locally asymptotically normal at all points f n 6 S i n the direction H(f ) with 
normal izing factor A,,(/ n ) (see the deta ils in Ivan der Vaartl (bood) "). Then, by the re- 
sults of llbragimov & Khas'minskiil (I1991D say that under these conditions, denoting K n = 

B n 9'(fo)A n Pr H (f ) with B n = y/nu, if K n — > K and if K{u) = (t,u), then for every esti- 
mator T>™' of Tij(f) and every family V(/o) of vicinities of f , we have 



inf liminf sup nE\Tp - T tj (f )] > lit 

{V(/ )} n^oo /gV(/o) JV /J " 11 



I L 2 (dxidxjdy) ' 



Here, 



K n (u) = ^T'(fo) ■ y^Pr H{fo) (u) = T'(fo) (y/fi> [u-y/hj uy/jA 

since for any u e ~h 2 (dxidxjdy) we can write it as u = vTo (v7o, w) + Pr#(/ )(u). In this case 
K n (u) does not depend on n and 

= T'(/ ) • ( L-y/Ef h^h\ ) 



with 



t(xi,Xj,y) = H 1 (f ,x i ,x j ,y)y/fo - (J ffi(/ ,Xi,Zj,j/)/ J a//o- 
The semi-parametric Cramer-Rao bound for this problem is thus 



and we recognize the expression CV,(/o) found in Theorem [TJ 



□ 



Proof of Corollary\l] The proof is based in the following observation. Employing equation 
([24]) we have 

T (n) -T(f) = Z^(f) + 



n 



where Z (n) (/) and R {n) are matrices with elements and R§\ defined in ([23]) and ([24D , 
respectively. 
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Hence we have, 

nE[||vech (f W - T(f) - Z<»>(/)) || 2 ] = E[||vech (flW) || 2 ] = ^[(i^V] 



i<3 



We see by Lemma that E[i?? ] ->■ as n ->■ 0. It follows that 

nE[||vech (t W - T(/) - Z (n) (/)) || 2 ] -»■ as n ->• 0. 

We know that if X n , X and Y n are random variables, then if X n — > X and (X n — Y n ) — > 0, 

follows that Y n X. 

Remember also that convergence in L 2 implies convergence in probability, therefore 

v^vech(f (n) -T(/)-Z(")(/)) AO. 

By the multivariate central limit theorem we have that y/n vech (z^ n \f) J — — >■ A/" (0, C(f)). 
Therefore, Vnvech ff (n) - T(/)) A jV(0, C(/)). □ 

Proof of Theorem^ For abbreviation, we write M instead of M n and set m = \M n \. We 
first compute the mean squared error of 6 n as 

E [9 n - 9} 2 = Bias 2 + Var (§ n ) 

where Bias (0 n ) = E[0 n ] - 6. 

We begin the proof by bounding Var (§ n ) . Let A and Bbemxl vectors with components 



6/ = / Pi(xa,Xj 1 ,y)f(x i 2, x j2 , y)^(xi U x jl} x i2 , x j2 , y)dx il dx jl dx i2 dx j 2dy 
pi(xi, Xj, y)g(xi, Xj, y)dxidxjdy I = 1, . . . , m 



where g(xi, Xj, y) = f f(xi 2 , Xj 2 , y)ip(xi, Xj, Xi 2 , Xj 2 , y)dxi 2 dxj 2 . Let Q and R be m x 1 vectors 
of centered functions 

qi(xi, xj, y) = pi(xi, xj, y) - a t 

ri(xi, Xj, y) = Pi{x i2 , x j2 , y)^{xi, Xj, x i2 , x j2 , y)dx i2 dx j2 - b t 



for I — 1, ... ,m. Let C a m x m matrix of constants 

c w = J Pi{xn,Xji, y)p v {x i2 , x j2 , y) r q(x il , x j2 , y)dx il dxj 1 dx i2 dxj 2 dy I, I' = 1, . . . , m. 
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Let us denote by U n the process 

n 

Unh=—±— h{x\ k \xf\Y*\x\ k '\xf'\Y^) 
and P n the empirical measure 

n 

p n h = - y £h{x?\xj k \Y^) 

k=l 

for some h in h 2 (dxi, dxj, dy). With these notations, 9 n has the Hoeffding's decomposition 

1 n 

*• = r7~TTT E E W^'. yW > + <*) W^, ^'') + M 

^ ' leM k^k'=l 

1 n 



n(rt — 1) ' ' 

v y M'eM k^k'=l 

U n K + P n L + A T B - A T CA 



where 



K (x a , Xjt, y u x i2 , x j2 , y 2 ) = Q T (x il ,x jl , yi)R(x i2 , x j2 , y 2 ) - Q T (x a , x n , yi)CQ(x i2 , x j2 , y 2 ) 
L(xi, Xj, y) = A T R(xi, xj, y) + BQ{x h xj,y) - 2A T CQ(xi, xj, y). 

Therefore Var(# n ) = Var (U n K) + Var (P n L) — 2 Cov (U n K, P n L) . These three terms are 
bounded in Lemmas [2] - [4] which gives 



n(n - 1) 11 MooM llo ° XtXjX ' n 
For n enough large and a constant 7 e M, 

v«W<7iHiLll/t^«(^+s)- 

The term Bias (§ n ) is easily computed, as proven in Lemma [5l is equal to 



(S M f(xn,Xji, y) - f(x a , x jU y)) (S M f(x i2 , x j2 , y) - f(x i2 , x j2 , y)) 

rj(xn, Xji,x i2 , x j2 , y)dx il dx j idx i2 dxj 2 dy. 

From Lemma [H the bias of 9 n is bounded by 

Bias(0 n ) <A ||r/|| sup|q| 2 . 
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The assumption of (sup l( £ M |Q| 2 j ^ m/n 2 and since m/n — > 0, we deduce that E[9 n — Q\ 

has a parametric rate of convergence O (1/n). 
Finally to prove ([TBI , note that 

nE [§ n - 9] 2 = n Bias 2 (d n ) + n Var (0 n ) 

= n Bias 2 (0 n ) + n Var (C^if) + n Var (P n L) . 

We previously proved that for some Ai, A 2 G M 



Bias 2 (^ t )<A 1 A 2 IHI 2 ^ 



2 II m2 m 



nV a T{U n K)<X 2 Al iX] \\f\\Jr ] \ 

I L 

Thus, Lemma [6] implies 

, Var(P n L) - A(/, 77) ] < A [\\S M f ~ f\\ 2 + \\S M g ~ g\\ 2 ] 



\n 



where A is a increasing function of Ij/H^, H^H^and A . From all this we deduce (TlBl 
which ends the proof of Theorem [3l □ 



6 Technical Results 

Lemma 1 (Bias of #„) . The estimator 6 n defined in (fT4l ) estimates 6 with bias equal to 



- J (S M f(xn, x n , y) - f(x a , x n , y)) {S M f(x i2 , x j2 , y) - f(x i2 , x j2 , y)) 

r}(xn, Xj 2 , y)dxndxj\dxi 2 dxj 2 dy. 

Proof. Let § n = l n -6 2 n where 

^ ' l£M k^k'=l J 

1 n 

n n — 1 ^— ' z — ' J J 

v 7 1,1' eM k+k'=\ 

\ pi(x a , Xji, y)p v (x i2 , x j2 , y)r)(x a , x j2 , y)dx il dx jl dx i2 dx j2 dy. 
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Let us first compute E [0*] . 

E [®n]=^2 / Pi( x n> x ji>y)f( x ii> x ji>y) dx ii dx jidy 

■ Pl{xa , yWxa , „, y Wxi2 , y MlixM 
^2ai Pi(xn,Xjh ytyfau Xji, x i2 , Xj 2 , y)f(xi2, Xj 2 , y)dx i idx j idxi2dx j2 dy 

/ ( ^(^2, Xj2, y) J ip(xn, xji, x i2 , x j2 , y)f(x i2 , x j2 , y)dx il dx j idxi2dx j2 dy 
\ieM J 

S M f{xn, xji, y)f(x i2 , x j2 , y)v(xii, x j2 , y)dx i idx j idx i2 dx j2 dy 
Now for we get 



Xj, y)f{xi, Xj,y)dxidxjdy 



E K] = / Vi(, x ii x jiy)f( x h x j,y)dxidx j dy / ; 

Pz(x a , Xji, y)pi'(x i2 , x j2 , y)v( x n, x j2 , y)dx il dx j idx i2 dx j2 dy 



1,1'eM 



/ I y^P^a^ji)?/) I I X] a vVv{xi2,Xj 2 ,y) J r](x i i J x j2 ,y)dx il dx j idxi 2 dx j2 dy 
SAiffauXj!, y)S M f(x i2 , x j2 , y)rj(xn, x j2 , y)dx il dx jl dx i2 dx j2 dy. 



Arranging these terms and using 

Bias (§ n ) = E[e n ] - e = E[§i] - E[e 2 n ] - e 

we obtain the desire bias. □ 
Lemma 2 (Bound of Vax(U n K)). Under the assumptions of Theorem\3\ we have 

Var(^) < ^^ylhllLll/llLA^.Cm + l) 
Proof. Note that U n K is centered because Q and R are centered and (x\ k \xf\ k = 
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1, . . . , n is an independent sample. So Var([y" n X) is equal to 



n n 



VW 71 - 1 )) fcl ^=i fc2 ^=i 
*:(xf 2) , xf 2) , y( fc2 \ xf a) , xf \ y(^) 

= — ^— -E f x 2 (x« , x, (1) , , xf > , xf , y( 2 ) ) 
+x(xf f xf , y« xf >, xf >, y( 2 ))x(xf \ xf >, y< 2 \ xf >, xf , yW) 

By the Cauchy-Schwarz inequality, we get 

Var(^X) < ^ryyEfX 2 (xf, xf , y« Xf >, xf , ^)] . 

Moreover, using the fact that 2 |E[Xy] | < E[X 2 ] + E[y 2 ], we obtain 



e[x 2 (xf,xf ^xf ,xf ,y( 2 ))] <2 



E[(g T (x«,xW,yW)i?(xf ,xf ,y( 2 ))) 2 ] 
+E[(g T (xf ) ) x| 1) ,y( 1 ))cg(xf ) ,xf,y( 2 ))) 2 ] 



We will bound these two terms. The first one is 

^l{Q^X?\xf\Y^)R(X?\xf\Y^)) 2 ] 

i,i'eM ^ 



P«0i2, 2^2, y)pi'(x i3 , x j3 , y)i)(xa, x n , x i2 , x j2 , y) 
, Xj\ ) Xj3, XjSi y)f(x il ,x jl ,y)dxi 1 dx jl dxi2dx j 2dx i3 dx j3 dy - bib v 

=Wi - ty 2 - w 3 + w 4 
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where 

J 1,1'GM 

i/}(x iA , x jA , x i3 , x j3 , y')f(x a , x n , y)f(x i4 , x j4 , y')dx il dx jl dxi2dx j2 dxi 3 dx j3 dx ii dx ji dydy' 

W 3 = ^ a l a l'Pl(Xi2,Xj2,y')Pl>(Xi3,Xj3 } y') 
J 1,1' GM 

Xj4, Xi2, Xj2, y , Xj 4 , Xj3, Xj 3 , y )f{x i4 , x j4 , y')dxi2dx j 2dx i3 dx j3 dx iA dx j4 dy' 

Wi= ^ aia v bib v . 



1,1'GM 

W 2 and W 3 are positive, hence 



E[(2Q T (Af >,Xj 1) ,y< 1 ))fl(Af >,4 2 >,1'< 2 ))) 2 ] < W, + W. 



J Li'eM 



1 J2 



ix j3 I /(x a , Xji, y)f{x i4 , x j4: , y^dxndxjidx^dxjidydy' 

-ll^lll / Vii,xn,x j i,y)p v (x i i,x j i,y)dx il dx j idy 
i,i'eM 

Pi'(x i3 , x j3 , y')ip(x i4 , x j4 , x i3 , x j3 , y')dx i3 dx j3 ) dx^dx^dx^dx^dy' 




Since p/'s are orhonormal we have 



Wi <l|/|lly^ j ^ Pi(xi2,x j 2,y')tp(x i4: ,x j 4 : ,Xi2,x j 2,y')dxi2dx j 2^j dx i4 dx j4 dy'. 

Moreover by the Cauchy-Schwarz inequality and < 2||r/|| 

2 r 

>\2 



Pi(x i 2,x j 2,y')^(xi4,x j ^x i 2,x j 2,y')dxi2dx j2 ) < I pi(x i2 ,Xj2,y') dxi 2 dx j2 

tl>(xu, x j4 , x i2 , Xj2, y') 2 dx i2 dx j2 
^IMlL A ^- f Pi(xi2,x j 2,y') 2 dx i2 dx j 
- 4 IMlL A ^ f Pi(x l 2,x j 2,y') 2 dx l2 dx 
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and then 




Pi(x i2 , Xj2, y')ip(xiA, Xj±, x i2 , Xj 2 , y')dx i2 dx j2 \ dx i4: dx jA dy' 



- 4 IMlL A ^ J Pi(x i2 ,x j2 ,y') 2 dx i2 dx j2 dy' 



= 4|h|| 2 

II 'II OO •*j%<Jjj 

Finally, 

For the term W± using the facts that Sm} and Sji/p are projection and that J f = 1, we have 



^< 4 IMlLll/||>W"- 



2ii i|2 II .11 || m2 
2ll^ll2 — P llooll^lh' 



By the Cauchy-Schwartz inequality we have ||g|| 2 <4||?7|| 2 II /II Al and then 

II I I II II OO II II OO t J 

w * <4v\\ 2 \\f\\ 2 A'-x- 

— II 'MOO II*' MOO X z Xj 

which leads to 

E^xfUjV^i^ (25) 
The second term E[(Q T (xf \ X^, Y^)CQ{xf \ xf\ Y^))] =W 5 - 2W 6 + W 7 where 

W 5 = J ^2^2 c hi[ c hi'JPh{xiux jU y)pi 2 (x a , x jU y)pi' 1 {x i2 , x j2 , y')pi> 2 (x i2 , x j2 , y') 

f(x a , x n , y)f(x i2 , x j2 , y^dxadx^dx^dx^dy'dy 
W 6 = ^^ci 1 i' i c h i' 2 ai 1 ai 2 pi' i {xi,x j ,y)pi' 2 {xi,x j ,y)dx i dx j dy 

h,l[ h,l' 2 

W 7 = y y^ c hl!i c hV9 a h a^ a h ay . 

Using the previous manipulation, we show that W e > 0. Thus 

m[(Q T {xP,xf\Y^)CQ{X?\xf\Y^))] <W 5 + W 7 . 
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First, observe that 

W 5 = ^ ^ c ilt / q 2 ;/ ^ pi 1 (x a ,x jl ,y)pi 2 (x il ,x jl ,y)f(x il ,x jl ,y)dx il dx jl dy 

-ML X] c «i c W 2 yj Ph( x iu Xji, y)pi 2 {xa, x jU y)dx il dx jl dyj 

Pi[(xi2,Xj2,y')pi' 2 (xi2,Xj2,y')dXi 2 dx j2 dy'] 
f\\ 2 Vc 2 , 



l,l> 



agin using the orthonormality of the the pfs. Besides given the decomposition pi(xi, x 
ai a {x i} x j )f3i p (y), 

1,1 Ipjp 

^ {^j ai a (x i i,x jl )a l i a (xi2,x j2 )ri(x il ,x j 2,y)dxi 1 dx jl dxi2dx j 2^j 
{^J on a {x i3 , x j3 )ai' a (xi4, Xj4)r](xi3, x j4 , y')dxi 3 dx j3 dx i4 dxj^j dydy' 

But 



^ (/ a ^( X il^ X jl) a l'A X ^^ X ^)v( X ily X j2,y)dx il dx j 

Oii a (xi3, x j3)oii' a (xiA, x j4 )r](xi 3 , Xj4, y')dx i3 dx j3 dx i4 dx j4 

^ / ai a (xi 1 ,x jl )ai /a (xi2,x j 2)r]( x ii, x j2,y)ai a (x i3 ,x j3 ) 

ay a (xj4, a^) 77 (2^3, Xj4, y')dxndxjidxi2dxj2dxi 3 dxj 3 dxi4dxj4 
J^(^J ai a (x il ,x jl )r](x il ,x j 2,y)dx il dx j i^j ai a (x i3 ,x j3 ) 

y^ (/" ai 'S x i^ x i^( x tti x j^y')dxi4dxj^ ai> a (xi2, Xj 2 )dxi2dxj2dxi 3 dx j3 



< J r)( x i3 y)ri(x i3 , x j2 , y')dxi 2 dxj2dxi 3 dx j3 

< A;.,. |//|| 2 
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using the orthonormality of the basis a la . Then we get 



< a UMII£(/a>)*/ 



b 

< Al.„. \\ri\\ 2 m 



since the f} x are orthonormal. Finally 

W 5 < ll/ll 2 |M| 2 A^ 

u — IK llooll 'Moo ^i-lj 

Now for W 7 we first will bound, 



m. 



/] c w aia v 
i,v 



22 a>ia>i'Ph{%iU Zji, y)Pi[(xi2, Xj2, y)v{ x ih x j2, y)dx il dx jl dx i2 dx j2 dy 



Ll'eM 



< / \Sm(xh, Xjh y)S M (xi2, x,2, y)r)(xn, x j2 , y)\dx iX dx jX dx i2 dx i2 dy 



— M 




\S M (xii, Xji, y)S M (x i2 , x j2 , y)\dy) dx iX dx iX dx i2 dx i2 



Taking squares in both sides and using the Cauchy- Schwartz inequality twice, we get 



^ c iv aia v 
i,v 



< 



< 



< 




\S M (xa, Xji, y)S M (x i2 , x j2 , y)\dy) dxadx^dx^dx^ 



' I I OO <*j%Jj^ 




\SM(xn,Xji, y)S M (x i2 , Xj 2 , y)\dy J dxiidxjidx i2 dx j2 
1 A; ,. j (^j S M {xiuXji,y) 2 dyj S M {.Xi 2) x j2) y') 2 dy^j dx iX dxjxdx i2 dxj 2 
T^H^A^. J S M (xa, Xjt, y) 2 S M (xn, Xji, y' fdxudx^dx^dx^dydy' 
^\\lo A ^j (J Suix^x^yfdxidxjdyj 



II 2 A2 II f\\ 2 
' II OO -Ei&j IK 1 1 oo 



Finally, 



^[{Q\X^\xf\Y^)CQ{xf\xf^)) 2 ] < \\4l\\f\\lAl( m + l). 



(26) 
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Collecting fl25]) and (1261) . we obtain 

Var(EWT) < ^^IMlll|/|lX,> + l) 

which concludes the proof of Lemma El □ 
Lemma 3 (Bound for Var(P n L)). Under the assumptions of Theorem\3\ we have 

Var(P n L) < -|U| 2 ll/ll 2 A 2 

\ ) „ II 'II OO II*' II OO x l x ] 

Proof. First note that given the independence of (X- k * , X- , yW) k = 1, . . . , n we have 

Var(P^) = i Var(L(xfUjV (1) )) 
we can write L{xf~\ xf\ F«) as 

A T R ,y«) +B T Q (x\ 1 \xf\Y^)-2A v CQ (x\ l \xf\Y^\ 

= J2 ai ( [ X J> z i. x\ x \xf\ Y^)dx l dx J - 6| ) 

+ J2b l ( Pl (X?\x! j 1 \YM)-a l )-2 c lv a v { Pl {Xf\xf\Y^)-, 
leM i,i'eM 

leM 

+ J2b l p l (X?\x?\Y W ) - 2 J2 cwa m (xP,X^,Y^) - 2A*B - 2A t CA. 



leM i,i'eM 

S M f(x h Xj , Y^{x h Xj , X[ l \xf\ Y^)d Xi dx 3 + S M g(x\ l \ xf\ F«) 

-2 c lv a vVl {x\ 1 \xf\Y^)-2A~ v B-2A T CA. 
1,1'eM 

Let h(xi, xj, y) = f S M f(x i2 , x j2 , y)4>{x h Xj,x i2 , x j2 , y)dx i2 dx j2 , we have 
S M h(xi,Xj,y) 

= ( / h ( Xi2 > x i 2 ' y)Pi( x i2, x j2, y)dx i2 dx j2 dy J pi{x h Xj, y) 
leM ^ ' 

= ( / S Mf(%i3, Xj3, y)ip{xi2, Xj2, x i3 , x j3 , y)pi(x i2 , x j2 , y)dx i2 dx j2 dx i3 dx j3 dy J pfa, Xj, y) 
leM ^ ' 

= ^ f / aifpi>(xi 3j x j3j y)^(xi 2 ,x j2 ,Xi 3j x j3j y)pi(xi 2 ,x j2 ,y)dx i2 dx j2 dxi 3 dx j3 dy 
i,l'eM ^ ' 

= 2 Y ( / a l >pi,{x i3 ,x j3 ,y)ri(x i2 ,x j3 ,y)pi{x i2 ,x j2 ,y)dx i2 dx j2 dx i3 dx j3 dy) pi(x h Xj,y) 
i,i'eM ^ ' 

= 2 Y a i' c a'Pi{ x i, x j,y) 



l,i'eM 
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and we can write 

L(X?\xf\YU)=h(X?\x?\YV) + S M g(X?\xf\YU) 
-S M h{x\ 1 \xf\Y < ^) - 2A T B - 2A T CA. 

Thus, 

= Var^^W, Xf\ K«) + S M9 (XP,XM, F«) + S M h(X?\ xf\ 

^[(fctxfUf,^) +s M h{x\ l \xf\Y^)) 2 ] 

<E[(/ i (X^xjV (1 % 2 +(^ 
Each of these terms can be bounded 

E[(/>(xfUjV (1) )) 2 ] 
= j {^J SMf(xi2,x j 2,y)ip(x il x j 2,Xi2,x j 2,y)dxi2dx j ^j f(x i i,x j i,y)dx a dx j idy 

< A x. lXj / S M f{x i 2,x j 2,y) 2 i){x i\Xj2i Xi2, Xj2, y) 2 f{xn, x n , y)dx il dx jl dx i2 dx j 2dy 

=4A ^ll/ILIhllLll' s ^ll2 

<4A 2 .J|/|| Ml 2 ll/ll' 



<4AL.ii/ir h 

X x Xj 1 1 d 1 1 oo I I ' . . . 

and similar calculations are valid for the others two terms, 



ll/l 


Lll^lla ^ 


1 1 ^ 1 1 cx) 1 1 ^ 1 


| 2 <4A 2 . .1 

1 2 — X{Xj \ 


1 f II 2 II ll : 

1 ^ 1 1 oo 1 M 1 1 c 


ll/l 


LIMII^ 


11/11 IN 

IK lloo II 1 


| 2 <4A 2 . . 

1 2 X{Xj 


ll/|lllMI 



oo 



E[(S M / 1 (lf ) ,ljV (1) )) 21 ' 
Finally we get, 

Var(P n L) < ^Ihtll/tA^. 

Lemma 4 (Computation of Cov([/ n i^, P n L)). Under the assumptions of Theorem\3\ we 

Cov (U n K,P n L) =0. 
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Proof of Lemma^ Since U n K and P n L are centered, we have 

Gov (U n K,P n L) 
= E[U n KP n L] 

1 



n 2 (n — 1) 



K{x[ k \xf\Y^ k \x\ k '\xf'\Y^)Y,L{x\ k \xf\Y^) 



k=l 



-E [K (xf \ xf\ y« , xf >, xf \ Y ^ ) (L (xf \ xf, y«) + L (X? \ xf \ Y^))} 



±E[(g T (xf , xf , r«)i?(xf \ xf , y( 2 )) - g T (xf , xf , y«)cg(xf , xf , y( 2 ))) 

Td/vW v(i) v(i)\ , nTnMi) y(!) v(i)^_ojT m /y(i) 



(A T i?(xf , xf , yW) + 5 T g(xf , xf , y«) - 2A T cg(xf , xf , y«) 
+a t r(xP , xf , y( 2 )) + B r Q{x?\x?\ y (2) ) - 2A T CQ{x?\xf\ y (2) ))] 

= 0. 

Since X, L, g and i? are centered. 

Lemma 5 (Bound of Bias (0 n )). Under the assumptions of Theorem\3\ we have 



□ 



Bias (0„) < A M sup |cf. 



Proof. 



Bias (# n ) 



— M 




I^m/^, ^-2, 2/) - f(x a , x j2 , y)\dx i2 dx j2 dy 




I \SMf(xi,Xj,y)-f(xi,Xj,y)\dxidXj) dy 
- A *<*JHL J ( S Mf(xi, Xj, y) - f(x h Xj, y)) z dx { dxjdy 
= A XiXj \\ri\\ 00 ^2 aiav I Pi(x l ,x j ,y)pi>(x i ,x j ,y)dx i dx j dy 

l,l'<£M 



14M 



14M 



□ 



We use the Holder's inequality and the fact that / e £ then Y1i4m \ a i\ — su Pzp/ \ c i\ ■ 

Lemma 6 (Asymptotic variance of y/n(P n L).). Under the assumptions of Theorem \3\ we 
have 

nV^(P n L)^A(f, V ) 

where 

Mf,v)= / g{x u x j ,y) 2 f(x h x j ,y)dx i dx j dy - ( / g(x i ,x j ,y)f(x i ,x j ,y)dx i dx j dy) . 
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Proof. We proved in Lemma [3] that 

Var(L(xfUf\y«)) 
= VK(h(X?\xj 1 \YV)+S u g(xi 1 \x< 1 \ 
= Vai(A 1 + A 2 + A 3 ) 

3 

= J2Cov{A k ,A l ). 

k,l=l 

We claim that Vk, I G {1, 2, 3} 2 , we have 



Cov(A fc ,A,) -e M 



where 



<*[pHf-f\\, + pug-g\\A (27) 



■1 if /c = 3or/ = 3andfc^Z 
otherwise 



and where A depends only on II f II , IMI and A,, „. . We will do the details only for the 

^ II u II OO II ' I I CO J^t^j J 

case k = I = 3 since the calculations are similar for others configurations. 



Var(A 3 ) = j Sj t h(x i ,x j ,y) f(x i ,Xj,y)dx i dx j dy-\^j S M h(xi,Xj,y) f(x i ,x j ,y)dx i dx j dyj . 

The computation will be done in two steps. We first bound the quantity by the Cauchy- 
Schwartz inequality 



S M h (xi, Xj,y) f(xi, Xj, y)dxidxjdy - j g(x h x 3 -,y) f(x i} x j ,y)dx i dx j dy 
<J\Sl,h (*„ ,„ y) /(,„ ,„ y) - Sly (x„ ,„ „> /(«,, ,„ „) | dz.d^dy 
+ / l^tfOEi.a^j/)/^,:^,^) - g(xi,Xj,y) 2 f(xi,Xj,y)\(kcidxjdy 



- \\f \\oo\\ SMh + ^Mfi'iyi^M^ - <Sju0|| 2 + H/H^pMfi' + #|| 2 ||SW - g\\ 2 . 

Using several times the fact that since Sm is a projection, ||5 , a/5 , || 2 < ||p || 2 , the sum is 
bounded by 

ll/lLll^+^iyi^-^iL+sii/iLii^iyi^-^i^ 

<||/|L(IHI 2 + b\\ 2 ) \\ h -9\\ 2 + 2\\f\\J\g\\ 2 \\SMg-g\\ 2 - 



We saw previously that \\g\\ 2 < 2A X;11 
is then bound by 

1 3/2 



1/2 

i x ' j I I ^ I I OO II ' I I OO 



II II II 1 1 1/2 1 1 || 

7? and \\h L < 2A„ „ f 77 . The sum 

x i x j\\ J I I OO II ' I I OO 



3/2 I 



I loo II 'II oo 1 1 a 112 i^jIK 1 1 oo || 'II oo II u u 
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We now have to deal with \\h — g\ 



\h - g\ 




{S M f(x i2 , Xj2, y) - f{x i2 , Xj2, y)) il>{xii,Xji,x i2i x j2 , y)dx i2 dx j2 j dxudxjidy 

< 4A L J lhllLl|s«/-/||'- 



Finally this first part is bounded by 



Sl,k («,, ,„ y) /(*„ X „y) W y - j 9 (*„ *„„)»,<*, x„ y) W y 

£ 4A I «ll/|iril"L( 2A ^ J ll"ILII s «/-/|| 2 + ll s »'9-9| 

Following with the second quantity 



S M h (Xi, Xj, y) f(xi, Xj, y)dxidxjdy 



g(x h xj, y)f(x h xj, y)dxidxjdy 



(S M h (xi, Xj, y) - g(xi, xj, y)) f(x h Xj, y)dxidxjdy 



(S M h (x^ Xj, y) + g(xi, Xj, y)) f(x u Xj, y)dxidxjdy 



By using the Cauchy-Schwartz inequality it is bounded by 

ll/IUI^A/^ - ^IMI/IUI 5 '^ + s\\ 2 

-II/II2 (IHI2 + IMI2) {\\ s Mh - S M g\\ 2 + \\S M g - g\\ 2 ) 



<4A„ 



3/2 1 



^IL (ll /l_ ^ll 2 + \\ S m9-9 



< 4A ^ \\f\t 2 h\L { 2 Kjv\\JSMf - f\\ 2 + \\S„g -g\ 

using the previous calculations. Collecting the two inequalities gives ([271) for k = I = 3. 
Finally since by assumption V£ G L 2 (<i/i), ||Sa/£— 1|| — > when n — > 00 a direct consequence 
of d27D is 

lim Var(L(X« xWyW)) 
5f 2 (x i; Xj,y)f(xi, Xj, y)dxidxjdy - (J g(x u Xj, y)f{x i} x j ,y)dx i dx j dy 

We conclude by noting that Var{y/ri(P n L)) = Var(L(X i (1) ,x| 1) ,FW)). □ 
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Lemma 7 (Asymptotics for y/n(Q — Q) ). Under the assumptions of Theorem\T] we have 

lim nE\Q-Q} 2 = 0. 
Proq/: The bound given in (fT6l) states that if |M n | /n — > we have 



ni[(g-g)V] - 

— ^dl/ILl l l ^ l l oo ' ^XiXj ;) 



g(x i ,x j ,y) 2 f(x i ,x j ,y)dx i dx j dy- ( / g(x h x j ,y)f(x i ,x j ,y)dx i dx j dy 
\M, 



n 



+ S'a// - / L + Sm§ - g\ 



where g(xj, x.,-, y) = / H 3 (f, x it xj, x i2 , x j2 , y)f{x i2 , x j2 , y)dx i2 dx j2 , where we recall that H 3 (f, x a , x n , x i2 , x 



H 2 (f , Xn , Xj2 1 j X{ 2 1 Xj\i y) with.H 2 (f,x il ,x j2 ,y) 



m 



■,(f,y)). By deconditioning we get 



/ f(xi,Xj,y)dxidx. 



(xn-mi(f,y))(x 



J"2" 



nE[(Q-Qy] -E 

— ^ ( l l ^ l l OO 5 ll^lloo' ^XiXj ) 

Note that 



g(xi,Xj, y) 2 f(xi, Xj,y)dxidxjdy 
\M n \ 



g(xi, xj, y)f(xi, x j ,y)dx i dx j dy 



ii 



\S M f - /|| 2 + E[||Sm£ - g\\ 2 ] 



®[\\SmJ - g\\ 2 ] <V[\\S M g ~ S M g\\ 2 ] + E[\\g - g\\ 2 ] + E[\\S AIn g - g\\ 2 ] 
<2E[||^-^|| 2 ] +E[\\S Mn g-g\\ 2 ] 

where g(x ij x jl y) = J H 3 (f, x h Xj, x i2 , x j2 , y)f(x i2 , x j2 , y)dx i2 dx j2 . The second term con- 
verges to since g e E 2 {dxdydz) and Wt £ h 2 (dxdydz), J (S M t — t) 2 dxdydz —> . Moreover 



\9-9\\ 2 




y) - g(xi, Xj,y)} dx { dxjdy 



H 3 (f, x h xj, x i2 , x j2 , y) - H 3 (f, x h Xj, x i2 , x j2 , y) ) f(x i2 , x j2 , y)dx i2 dx j2 



dxidxjdy 



< 




H 3 {f i Xj, Xj, Xi 2 , Xj 2 , y) — H 3 (f, Xj, Xj, Xi 2 , Xj 2 , y) J dxi 2 dx 



J 2 



f(x i2 , x j2 , y) 2 dx i2 dx j2 



dxidxjdy 



- A x lXj \\f\\l I [H 2 (f,x u Xj,x i2 ,x j2 ,y) - H 2 {f,x i ,x j ,x a ,x j 2,y)) dx t dxjdx i2 dx j2 dy 



^ s ^x i x j \\f\\ 00 I [f(xi,Xj,y) - f(xi,Xj,y)) dx { dxjdy 



for some constant 5 that comes out of applying the mean value theorem to H 3 (f, Xi,Xj, x i2 , x j2 , y) — 
H 3 (f,x.i,Xj,x i2 ,Xj 2j y). The constant 5 was taken under Assumptions [T]l3l Since — 
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/|| ] — > then E[||<7 — g\\ 2 ] — >• 0. Now show that the expectation of 

j g(x i ,x j ,y) 2 f(x i ,x j ,y)dx i dx j dy~ (j g(xi,x j ,y)f(x i ,x j ,y)dx i dx j dy 
converges to 0. We develop the proof for only the first term. We get 

g(xi, xj, y) 2 f(xi, xj, y)dxidxjdy - J g(x, h Xj, y) 2 f(x i} Xj, y)dxidxjdy 
< I \g(xi,Xj,y) 2 - g(xi,Xj,y) 2 \ f(x i ,x jl y)dx i dx j dy 



<X J (g(xi,Xj,y) - g(xi,Xj,y)) dxidxjdy 
\\\" II 2 

for some constant A. By taking the expectation of both sides, we see it is enough to show 
that — g\\ 2 2 ] —^0. Besides, we can verify 

g(xi, Xj, y) = j H 3 (f, x h Xj, x i2 , x j2 , y)f(x i2 , x j2 , y)dx i2 dx j2 

2 

= T77 w a ( Xi ~ ™ i( <y)) 

J J %j i y jUXiClXj 

Xj2f(x i2 , x j2 , y)dx i2 dx j2 - rhj{y) / f(x i2 , x j2 , y)dx i2 dx j2 



= 

which proves that the expectation of J g(xi, Xj,y) 2 f(x il Xj,y)dxidxj converges to 0. Similar 
computations shows that the expectation of (J g(xi, Xj,y)f(xi, xj, y)dxidxj) 2 also converges 
to 0. Finally we have 

lim nE\Q-Q} 2 = 0. 

□ 
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